The subject invention is a surface data acquisition, storage, and assessment system and more particularly a surface data acquisition, storage, and assessment system for detecting and quantifying similarities or differences between stored data and data collected from a scan.
There are many disparate fields in which human senses are used as measuring tools. These fields are often specialized, technical, or industrial, and yet workers in these areas still acquire and gauge data using one or more of their biological senses. For example, the farmer looks at and feels or squeezes produce to determine its ripeness, or a butcher views and presses on a cut of meat to determine its freshness. A geologist tries to identify a type of rock or soil by looking at its colors and by running his or her hands over it, feeling for surface roughness and grain size. During a delicate surgery, a pathologist receives a sample of tissue that must be identified rapidly using only what can be seen through a microscope. An art critic tries to authenticate a painting by looking at the brushstrokes and by feeling the surface texture. In the modern world of technology, even robotic devices depend on viewing and touching an object for identification, such as a robotic arm reaching into a box of assorted parts to identify and properly pick up a part for placement into a machine assembly. In each of these cases, the worker (human or machine) is using his/her/its senses to evaluate the external appearance or surface features of the object to detect and measure surface data, such as color, temperature, contour, shape, surface roughness or smoothness, and the like. Clearly, given the technical nature and great importance of many of these fields, a need exists for a system that can rapidly, accurately, precisely, and objectively acquire and measure surface data and automatically compare these data to a standard, should such a standard exist. If such a standard does not exist, then the creation of such a surface assessing system would greatly aid in its creation.
One process that can be utilized to acquire and measure surface data and compare these data to a standard is by pose estimation, particularly when the scan requires acquiring data of a three-dimensional (“3D”) object. Pose estimation is a process that determines the position and orientation of known objects in 3D scenes relative to a remote observer. Humans perform pose estimation on a regular basis. Anytime somebody picks up a pencil or parallel parks an automobile they are using pose estimation to determine how to orient their hand properly to pick up the pencil or to initiate a trajectory that provides the best opportunity for parking successfully. In these two cases, the pose is determined using visual sensing, i.e. stereo vision and tactile feedback, but pose can be also derived from audio, radar, and other measurements that provide relative 3D position. Accordingly, pose estimation plays a significant role in a human's ability to interact with its environment, whether that environment is static or dynamic.
Pose estimation has been used in some computer vision applications for robotic or autonomous systems, where the system attempts to perform operations that are natural to humans. These applications include, but are not limited to, object identification, object tracking, path planning, and obstacle avoidance. Potential applications using pose estimation can be as simple as an industrial robotic system identifying a particular part from a bin of many different parts for picking up and loading into a machine, or as complex as autonomous aircraft flying in formation while navigating a terrain, or a spacecraft performing autonomous rendezvous and docking with a non-cooperative spacecraft by identifying docking features, developing an interception plan, and executing the plan. These applications however all require real-time pose estimation. Further, such systems for object pose estimation typically require that various landmarks or features (such as points, lines, corners, edges, shapes, and other geometrical shapes) must be identified and selected. A pose can then be made and registration performed using such identified references. Accordingly, such a system requires an object to have pre-identified features. Further, such methods often have difficulty with objects having same features but with different dimensions. Care must be taken in selecting such features as some objects may have the identified features but different non-identified features which could result in error.
Clearly, pose estimation can be applicable for certain robotic or autonomous systems, it also has other applications such as surface alignment. For example, surface alignment takes 3D surface measurements of multiple instances of the same object with different poses relative to the observer and applies a rigid transformation to the measurements so that each instance has the same pose relative to the observer. Surface alignment allows for automatic comparison such as defect detection in high production factory settings if one of the instances serves as a “truth” model. It also allows for the generation of complete 3D surface measurements of an object by stitching multiple 3D surface measurements from varying viewpoints. Varying viewpoints are required to generate a complete 3D surface measurement, because some regions of the surface are always occluded by others. With pose estimation, surface alignment can be performed with no knowledge of the relative pose of each surface as long as some overlap exists between the individual 3D surface measurements.
Systems have been developed for use in assessing medical conditions using various imaging systems. Systems for use in analyzing skin and joints have used baseline images for comparing with a current scan of the skin or joint. Such comparisons usually operate by the physician observing the scans. Other systems have been developed to automatically make comparisons of scanned images, the patients joint or body part being scanned are required to be immobilized using a specialized mold or jig in order to ensure proper alignment of the images for registering points on the images for making proper comparisons. Such immobilization is difficult for certain body regions and makes scanning problematic if such scans are being done at different locations. Further, the patients may require different methods for immobilization making the process more complex, time consuming, and expensive. While it may be possible to automatically make comparisons of scanned images of a patient, such systems require precise positioning of the patients joint or scanned area which again is time relatively complex, consuming and expensive. Further, such systems often require that the scanner making the scan must be consistently aligned with and/or consistently positioned relative to the surface being scanned. Other systems have been developed that require the physician to make artificial references on the surface of the patient being scanned for registering to allow for the proper alignment of the images.
Many pose estimation algorithms exist in literature, but it has now been found that the spin-image pose estimation algorithm provides the most accurate results while being robust to sensor noise and placing no surface restrictions on the object of interest other than it must be pose distinct (i.e. unlike a sphere). It also places no restrictions on the measurement technology other than it must generate a 3D surface mesh. Although the spin-image algorithm is accurate, like other robust pose estimation algorithms, the algorithm is computationally complex and the time required to compute a pose estimate is relatively long. Unfortunately, the relatively long computational time makes it inadequate for the many engineering applications that require a robust real-time pose estimation algorithm.
The fundamental principal behind a spin-image algorithm is to provide an efficient method for representing and matching individual points of an object's surface. It should be understood that by comparing and matching spin-images one is actually comparing and matching surface points. This representation is called a spin-image. By matching the spin-images of surface points in an observed scene (scanned image) to the spin-images of surface point of the “truth” model (reference image), surface point correspondences can be established. It should be understood that the truth model can be a scan, CAD model, mathematically defined surface, and the like. This matching procedure requires that each scene spin-image be compared to all reference spin-images by determining the linear correlation between the spin-images called the similarity measure. This is one of the most time-consuming portions of the algorithm and until now makes the spin-image pose estimation algorithm impractical for many applications. For an example, a typical spin-image is a 16×16 pixel image. Therefore, the spin-image is represented by an array of 256 numbers or “counts” at each of the 256 squares forming a grid over the image. To check for matches of spin-images, the 256 numbers in the spin-image for each point in the scene image must be compared to the 256 numbers in each reference spin-image. If the 3D scene image consists of a million points, and the reference spin-image also contains a million points, therefore there are a million of these 256 comparisons that must be made (256 million comparisons to check if the spin-image for one point in the scene spin-image matches the spin-image for one of the points in the reference spin-image). If multiple scene image points are to be compared to the full set of reference spin-image points, then the number of comparisons must be multiplied by the number of scene spin-image points to be matched. Therefore, spin-images with a larger number of grid squares (such as 32×32) results in even more computations to compare spin-images. Unfortunately, as a result of such a large number of comparisons that must be made, this method of using spin-image comparisons cannot be used for real-time pose estimation and is therefore not practicable for many applications.
In view of the foregoing, it is apparent that a need exists for a surface data acquisition, storage, and assessment system for detecting and quantifying similarities or differences between collected data obtained from the object of interest and stored data and which can operate in a relative short amount of time and preferably in relative real time. Accordingly, a need also exists for a method or process that decreases the time required for calculating a pose estimate such as by the use of a spin-image algorithm, thus increasing its performance thereby making it more practical for many applications that require real-time operations. Further, a need exists for a system that allows objects to be scanned without the need for the object to be immobilized with a specialized mode or jig when scanned or the scanner to be in the same position relative to the object for each scan, thus placing no restrictions of how the object being scanned is positioned relative to the scanner.